Skip to content

Pubsub dashboards guide#3091

Open
splindsay-92 wants to merge 28 commits intomainfrom
pubsub-guides/dashboards-with-pubsub-guide
Open

Pubsub dashboards guide#3091
splindsay-92 wants to merge 28 commits intomainfrom
pubsub-guides/dashboards-with-pubsub-guide

Conversation

@splindsay-92
Copy link
Contributor

@splindsay-92 splindsay-92 commented Jan 12, 2026

Description

A pubSub guide focused on creating dashboards and visualisations.

The guide covers;

  • Architecture, channel design patterns and throughput and optimisations.
  • Authentication, HIPAA compliance and token managment.
  • Handling network disruptions, state monitoring and history.
  • Presence and occupancy for view counts etc..
  • Outbound ,essage streaming for audit/analysis purposees

Checklist

@coderabbitai
Copy link

coderabbitai bot commented Jan 12, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch pubsub-guides/dashboards-with-pubsub-guide

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Member

@paddybyers paddybyers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@m-hulbert This text contains a lot of AI giveaways, such as em dashes, and emboldened bulleted lists. Does (or should) our style guide say we aim to avoid these?


Ably applies rate limits to ensure platform stability. By default, channels accept up to 50 inbound messages per second. Enterprise plans can request higher limits for specific use cases. When working with high-frequency data sources, consider batching multiple updates into single messages to stay within these limits.

For example, data sources generating more than 50 updates per second could be batched into periodic publishes:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we suggesting doing this instead of server-side batching?

Copy link
Contributor Author

@splindsay-92 splindsay-92 Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was suggesting both. SSB as a means of keeping costs down, but batching publishes as a means of dealing with high frequency data sources that would otherwise break the inbound channel message limit.

Copy link
Contributor Author

@splindsay-92 splindsay-92 Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ignore me, this is incorrect. I checked how limits work in go, and I now think the implementation isn't quite correct in the coordinator. Having checked node, I can see the difference and will update this accordingly. I'll also confirm why there is a difference in go.

I also think the SSB docs themselves should be updated, so will make a PR, as they are a bit lax tbh.


The key benefit of server-side batching is that it reduces billable outbound message count, especially during traffic spikes. If your source publishes 10 updates per second and you have 1000 subscribers, without batching you'd have 10,000 outbound messages per second. With 500ms batching, messages are grouped into 2 batches per second, resulting in 2,000 outbound messages per second—a 5x reduction.

Unlike message conflation, server-side batching preserves all messages and message order. Every update is delivered, just grouped together for efficiency. This makes it suitable for scenarios where you need complete data, but can tolerate some latency in exchange for cost savings.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some nuances about changes to message ordering - order is preserved for messages of a given type, but ordering of messages vs presence messages, or live objects updates, etc, can change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some text addressing ordering of different message types in batching.


#### Pairing with persist last message

For state-based dashboards using delta compression, the [persist last message](/docs/storage-history/storage#persist-last-message) channel rule provides a means to store and query the latest state on the channel. When enabled, Ably stores the most recent message published to a channel for 365 days. New clients can then attach with `rewind=1` to immediately receive the last published state, or query it via history.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be clear on when we recommend this vs just persistence?

A dashboard is likely to be based on data that's consistently updated, so persist-last doesn't feel like the right solution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid point, I will re-work this section a bit to cover persist vs persist-last. I don't think persist-last is the wrong choice, but agree it's an incomplete choice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reworked this to focus on regular persistence, using last-message persistence for times when the default 30 days is insufficient.

@m-hulbert
Copy link
Contributor

@m-hulbert This text contains a lot of AI giveaways, such as em dashes, and emboldened bulleted lists. Does (or should) our style guide say we aim to avoid these?

@paddybyers it doesn't currently, but I have a doc that updates the contributing guide to an MDX focus so I'll include an update to the style guide with that. It's a good shout.

Copy link
Contributor

@m-hulbert m-hulbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Steven. Added some comments throughout;

  • I agree with Paddy's comment about common AI'isms.
  • I think we should have some placeholders for images in here.
  • Is it worth mentioning charts as well as dashboards (will be good for images, and its likely the same data in most, if not all cases).
  • In the opening sections we talk about critical vs fan engagement as the 2 types of dashboard throughout, but then some sections don't mention this again and refer to different types.


The most important decision you can make when developing a realtime dashboard is understanding the experience you want users to have and the criticality of the data being delivered. This will determine the architecture, feature set, and ultimately the impression your users leave with.

The key architectural decision is understanding which category your dashboard falls into. With Ably, you are not limited by technology—only by the user experience you want to deliver. Realtime dashboards typically fall into two categories, each with distinct requirements:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the spacing is off here technology-only vs technology - only (or better yet technology, only)


For healthcare applications, Ably is HIPAA-compliant and offers Business Associate Agreements (BAAs) for customers handling Protected Health Information (PHI).

Understanding which category your dashboard falls into—or whether it combines elements of both—is fundamental to making the right architectural decisions throughout this guide.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be in the opening section rather than underneath one of the options?

```
</Code>

The single channel pattern maximizes cost efficiency because Ably's fanout delivers each published message to all subscribers with a single outbound message charge per subscriber. There's no duplication of data across channels, and the architecture remains simple to reason about.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"simple to reason about"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reworded this to be a bit clearer that I'm saying a single stream is easy to manage and can be scaled from 1 to 100 million subscribers with ease etc..


For dashboards where only the current state over some time window matters—stock prices, sensor readings, vehicle positions—[message conflation](/docs/messages#conflation) delivers only the most recent value within each time window.

Configure conflation through [rules](/docs/channels#rules) in your dashboard. You'll specify a conflation interval and a conflation key pattern that determines which messages are considered related. Messages with the same conflation key within the time window are conflated together, with only the latest delivered. Multiple conflated messages are then delivered in a single batch at the end of the interval.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't offering an equivalent paragraph for SSB as this one. We should either include for both, or assume the link to config instructions is sufficient IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed


Conflation dramatically reduces costs when publishers send updates faster than users can perceive. If a price feed publishes 10 updates per second but your dashboard refreshes only each second, you're wasting 90% of your message budget on updates users never see. With 100ms conflation, you reduce outbound messages by 10x while showing users the most current data available.

### Configuring rules
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this may be redundant given my previous comment. A brief overview of each about the interval and conflationKey etc. with a link to their relevant docs is probably enough rather than a whole section in here describing how to use the dashboard.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, this was removed.

```
</Code>

Presence is powerful but expensive at scale. Every enter, leave, and update event generates messages delivered to all presence subscribers. For a channel with 1000 viewers all subscribed to presence, a single user joining triggers 1000 outbound messages. If users are frequently joining and leaving, this can quickly dominate your message costs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's odd to mention this dominating your message costs and then the section about handling it in a more cost-efficient manner is 2 sections below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flow I had in mind was mention presence is expensive -> talk about occupancy as an alternative -> if you really need presence at scale, here's how you might do it.

I've reordered this now though so we talk about presence at scale before occupancy :)


[Server-side batching](/docs/messages/batch#server-side) groups messages before fanout, dramatically reducing outbound message count during high-activity periods. This is particularly valuable when your data source publishes multiple updates per second.

To understand the impact, consider a live sports dashboard with 100,000 viewers and a data source publishing 10 updates per second during an exciting match:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be sending this high frequency of data in a sports dashboard? 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Messages/reactions would count as high-frequency, albeit likely from multiple clients. Also, I'd expect telemetry data to be high-ish frequency.

It was mostly to highlight the point about reducing outbound message costs, but I could mention something else if you think it's not a good enough fit? Perhaps a health care monitor, where heart rates of multiple patients are sent to the same channel?


If you're using [message conflation](#message-conflation-for-latest-value-scenarios) to optimize dashboard delivery costs, be aware that conflated messages are also what gets streamed to external systems. If you need complete data for analytics or audit purposes but want conflation for dashboard delivery, you may have to consider other publish patterns.

### Message format considerations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is worth noting, but we don't need to go into all this detail / have a specific section for it.


Non-enveloped messages deliver just the raw payload, which is useful when your downstream system expects a specific format or you want to minimize data transfer. However, you'll lose the channel and metadata context that enveloped messages provide.

## Production-ready checklist
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the following 2 sections are a lot of bullet points which I'm not convinced are all helpful. A lot of the next steps links are already linked throughout the guide too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've trimmed these down a lot now, so hopefully they are less overwhelming.

Copy link
Contributor

@m-hulbert m-hulbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty solid. I especially think this would benefit from the visuals too.

meta_keywords: "realtime dashboard, pub/sub, fan engagement, patient monitoring, IoT dashboards, data streaming, scalability, cost optimization"
---

Ably Pub/Sub is purpose-built for realtime data distribution at any scale. Whether you're delivering live sports statistics to millions of fans, streaming critical patient vitals to a nurse's station, or updating stock prices across thousands of trading terminals, Ably handles the complexity of message distribution so you can focus on your application.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "the complexity of message distribution" the most important thing we're handling here? Or is this a carry over from the data distribution guide?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of both perhaps, but I can change it to focus on the other side of things, like scaling websocket servers, handling failover, or low latency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or I could just simplify to something lkie

Ably handles the infrastructure so you can focus on your application.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it wouldn't hurt to call out a few things that are perhaps more important to this use case like message ordering or guaranteed delivery.


Despite the challenges of delivering these guarantees, Ably is designed to keep costs predictable. Using features such as server-side batching, delta compression, and efficient connection management, along with Ably's consumption-based pricing model, ensures costs are kept as low as possible, no matter the scale.

## Architecting your dashboard or chart
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe 'architecting your app'?


In these scenarios, the relationship is typically one publisher broadcasting to many subscribers. High message throughput with sub-second latency is usually sufficient, eventual consistency is acceptable and intermediate values can often be discarded to reduce outbound messages. Key considerations include:

* Cost optimization becomes critical due to message fanout
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bullets on lists please 🙂


For healthcare applications, Ably is HIPAA-compliant and offers Business Associate Agreements (BAAs) for customers handling Protected Health Information (PHI).

Understanding which category your dashboard falls into, or whether it combines elements of both, is fundamental to making the right architectural decisions throughout this guide.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this sit in the intro section, rather than at the bottom of criticality? Is it worth mentioning a blend of the two approaches in that intro section too?

```
</Code>

Per-entity channels enable you to apply different [capabilities](/docs/auth/capabilities) to each channel, ensuring that users can only subscribe to the data they're authorized to see. This pattern also provides natural isolation where a spike in activity on one channel doesn't affect others, and you can apply different optimization rules to different channel namespaces.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we mean by "optimization rules" here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be clear here and mention server side batching and conflation specifically.

* Never expose API keys in client-side code
* Always use token authentication, with tokens generated by your server based on the user's authenticated session
* Apply the principle of least privilege, granting only the capabilities each user actually needs and on a per-channel basis
* For compliance scenarios, use [integration rules](/docs/platform/integrations) to log channel activity for audit trails
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* For compliance scenarios, use [integration rules](/docs/platform/integrations) to log channel activity for audit trails
* For compliance scenarios, use [integrations](/docs/platform/integrations) to log channel activity for audit trails


Ably maintains a 2-minute message buffer for each connection. When a client reconnects within this window, any messages published during the disconnection are automatically delivered, ensuring no data is lost during brief network interruptions.

For longer disconnections, or when you need to backfill historical data, use the [history API](/docs/storage-history/history) to retrieve messages. This will require the use of a [persistence](#Pairing with message persistence) rule to ensure messages are stored for retrieval.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing a link here for persistence.

* Use occupancy for the aggregate viewer count that everyone sees, but enable full presence only for specific user groups who need to see individual identities.
* Enable server-side batching on presence events via [rules](/docs/channels#rules) to both reduce outbound message counts during high churn periods and allow much higher effective inbound presence event rates beyond the default 50 messages/second limit.

If you are operating presence at scale, consider splitting presence into a separate channel from your main data stream. This allows you to apply different optimizations and access controls to presence without impacting the primary dashboard data, and ensure that high presence activity doesn't interfere with the integrity of your main data stream.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you are operating presence at scale, consider splitting presence into a separate channel from your main data stream. This allows you to apply different optimizations and access controls to presence without impacting the primary dashboard data, and ensure that high presence activity doesn't interfere with the integrity of your main data stream.
If you are operating presence at scale, consider splitting presence into a separate channel from your main data stream. This enables you to apply different optimizations and access controls to presence without impacting the primary dashboard data, and ensure that high presence activity doesn't interfere with the integrity of your main data stream.


### Batching strategies for cost optimization <a id="server-side-batching"/>

As covered in [Cost efficiency with batching strategies](#cost-efficiency-with-batching-strategies), both server-side batching and message conflation can dramatically reduce message costs. The choice depends on whether you need to preserve all messages or only the latest values.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
As covered in [Cost efficiency with batching strategies](#cost-efficiency-with-batching-strategies), both server-side batching and message conflation can dramatically reduce message costs. The choice depends on whether you need to preserve all messages or only the latest values.
[Server-side batching and conflation](#cost-efficiency-with-batching-strategies) can dramatically reduce message costs. The choice depends on whether you need to preserve all messages or only the latest values.

Comment on lines 739 to 743
1. Navigate to your app settings and select the **Integrations** tab.
2. Create a new integration rule and choose your destination service (e.g., Apache Kafka, Amazon Kinesis).
3. Specify which channels to stream using a regular expression filter (e.g., `^vitals:.*` for all patient vitals channels).
4. Select the event types to capture: messages, presence, lifecycle, or occupancy depending on your needs.
5. Configure the destination connection details, such as the Kafka topic or Kinesis stream name.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the other guide, I don't think we need to include these steps here.

@m-hulbert m-hulbert requested a deployment to ably-docs-pubsub-guides-ey14hq February 12, 2026 09:13 Abandoned
@m-hulbert m-hulbert requested a deployment to ably-docs-pubsub-guides-ey14hq February 12, 2026 11:15 Abandoned
@splindsay-92 splindsay-92 added the review-app Create a Heroku review app label Feb 12, 2026
@splindsay-92 splindsay-92 marked this pull request as ready for review February 12, 2026 16:20
@splindsay-92 splindsay-92 force-pushed the pubsub-guides/dashboards-with-pubsub-guide branch from fb7cca6 to e1ac615 Compare February 12, 2026 16:27
@m-hulbert m-hulbert requested a deployment to ably-docs-pubsub-guides-ey14hq February 12, 2026 17:36 Abandoned
@splindsay-92 splindsay-92 force-pushed the pubsub-guides/dashboards-with-pubsub-guide branch from e1ac615 to 53b0039 Compare February 13, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-app Create a Heroku review app

Development

Successfully merging this pull request may close these issues.

3 participants